1. Field of the Invention
This invention relates to machine vision systems and more particularly to uses for advanced machine vision search tools that register patterns transformed by at least two translational and at least one non-translational degree of freedom.
2. Background Information
The use of advanced machine vision systems and their underlying software is increasingly employed in a variety of manufacturing and quality control processes. Machine vision enables quicker, more accurate and repeatable results to be obtained in the production of both mass-produced and custom products. Basic machine vision systems include one or more cameras (typically having solid-state charge couple device (CCD) imaging elements) directed at an area of interest, frame grabber/image processing elements that capture and transmit CCD images, a computer and display for running the machine vision software application and manipulating the captured images, and appropriate illumination on the area of interest.
Many applications of machine vision involve the inspection of components and surfaces for defects that affect quality. Where sufficiently serious defects are noted, a part of the surface is marked as unacceptable/defective. Machine vision has also been employed in varying degrees to assist in manipulating manufacturing engines in the performance of specific tasks. In particular, a workpiece held in a robot manipulator (end effector) can be guided to a target using a machine vision feedback procedure known as visual xe2x80x9cservoing.xe2x80x9d In general, a robot end effector, or associated workpiece held therein, is guided to a desired final target location, such as an attachment point for a component part, based upon relative movement of the end effector within the field of view of a machine vision system. The robot is programmed with a general set of movement instructions. The machine vision system verifies that the end effector is moving within a particular coordinate grid in the proper direction toward the final target location. Typically, the grid is based upon movement within a two-dimensional image plane viewed by the camera, and involves a predefined/preloaded outline for the end effector (i.e. the machine vision system looks for a specific shape and correlated that shape to a given position). The vision system then instructs the robot whether the prevailing movement is proper, and the robot is servoed thereby into a proper movement direction. A discussion of prior art visual servoing techniques and principles is provided generally in the paper A Tutorial on Visual Servo Control by Seth Hutchinson, Department of Electrical and Computer Engineering, The Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign; Greg Hager, Department of Computer Science, Yale University and Peter Corke, CSIRO Division of Manufacturing Technology, Kenmore Australia; published May 14, 1996.
This servoing approach is limited in that it typically only analyzes translational degrees of freedom (e.g. those in the image plane) generally perpendicular to the camera axis, and does not take into account any non-translational degrees of freedom such as the distance (z-axis directionxe2x80x94toward and away from the camera), which relates to relative scale of the image, and/or the relative rotation (angle xcex8) of the end effector within the field of view. This servoing approach is therefore limited where the end effector is not located at a predictable location, or where the positioning of the camera has changed.
The advent of increasingly faster and higher-performance computers, has enabled the development of machine vision systems that employ powerful search tools. Such search tools enable a previously trained/stored image pattern to be acquired and registered/identified regardless of its viewed position. In particular, existing commercially available search tools can register such patterns transformed by at least three degrees of freedom, including two translational degrees (x and y-axis image plane) and a non-translational degree (rotation and/or scale, for example). One particular implementation of an advanced search tool is the rotation/scale-invariant search (RSIS) tool. This tool registers an image transformed by at least four degrees of freedom including the two translational degrees (x and y-axis image plane) and at least two non-translational degrees (z-axis(scale) and rotation within the x-y plane about an axis perpendicular to the plane. Some tools also register more complex transformations such as aspect ratio (rotation out of the plane whereby size on one axis decreases while size in the transverse axis thereto remains the same). These search tools, therefore, enable a specific pattern within the field of view to be located within a camera field of view to be positively identified and located accurately within the vision system""s internal reference system (an x, y, z, rotation coordinate system, for example). The RSIS and other advanced search tools particularly allow for the identification and acquisition of patterns having somewhat arbitrary rotation, scaling (e.g. distancing) and translation with respect to the reference system. In other words, the tool is sufficiently robust to recognize a desired pattern even if it is rotated and larger/smaller/skewed relative to a xe2x80x9cmodelxe2x80x9d or trained pattern within the vision system.
In general, advanced machine vision tools acquire an image of a pattern via a camera and analyze the outline or a particular part of the pattern, such as a predetermined fiducial mark. The processing speed of the underlying computer in which the tool resides is sufficient to enable a very large number of real time calculations to be completed in a short time frame. This particularly enables the search tool to determine the coordinates within an image reference system for each analyzed point in the viewed area, and correlate these through repetition with a desired pattern. The search tool may map the locations of various points in the captured image to stored points in the model image. A pattern is registered if the mapping falls within accepted tolerances and parameters. Using various decision algorithms, the tool decides whether the viewed pattern, in a particular rotation and distance (scale) corresponds to the desired search pattern. If so, the tool confirms that the viewed pattern is, in fact, the pattern for which the tool is searching and fixes its position and orientation.
Machine vision systems having a three-degree-of-freedom, or greater, capability (such as RSIS) are available from a number of commercial vendors including Hexavision(copyright) from Adept Technology, Inc. of San Jose, Calif., and the popular Patmax(copyright) system from Cognex Corporation of Natick, Mass. Advanced machine vision search tools such as Patmax(copyright) also have the ability to take advantage of the previous known position of a search subject or target. This narrows the search area to positions relatively near the last known location. Therefore, searching is relatively faster on the next cycle since a smaller area is searched. In addition, these search tools can tolerate partial occlusion of a pattern and changes in its illumination, adding further to their robustness with respect to less-advanced machine vision approaches.
It is therefore an object of this invention to provide a system and method for servoing a robot that is more robust than prior techniques and capable of accurately positioning workpieces held by robots onto a target location regardless of orientation within the field of view. The system and method of this invention should operate with a variety of robot configurations and for a great variety of robot movement patterns in each of at least four degrees of freedom.
This invention overcomes the disadvantages of the prior art by providing a system and method for servoing a workpiece held in a robot end effector, operating within a work area, that uses an advanced machine vision search tool capable of registering an acquired pattern in an image transformed by at least three degrees of freedom including, generally, at least two translational degrees and at least one non-translational degree of freedom with respect to an image plane (e.g. x and y-axis plane) as viewed by the machine vision system. The machine vision search tool can be a rotation/scale-invariant search (RSIS) tool, or another equivalent implementation adapted to recognize and locate one or more previously trained and calibrated fiducial marks on the workpiece regardless of translation and rotation of the imaged fiducial within the specified degrees of freedom (four or more in this example). In this manner, the relative location of the workpiece can be more accurately, and readily determined regardless of variations in the rotation and positioning of the workpiece, allowing the workpiece position to be located and corrected to reliably locate the target.
According to a preferred embodiment the machine vision tool is adapted to locate a trained fiducial mark on the workpiece when the workpiece enters an area of interest within a field of view of a camera of the machine vision system. The fiducial is recognized by the tool based upon past training of the tool to recognize selected fiducials. The location of the workpiece is derived by the machine vision system based upon the found location of the fiducial. The found location of the fiducial is compared to the desired location of the fiducialxe2x80x94derived typically from a fixed position at which the workpiece is supposed to be within the area of interest (e.g. the target). If there is a difference between the found location and the desired location, then the difference is transformed using mapping software into an adjustment value within the robot""s coordinate reference system. The difference is generally resolved into values along a plurality of axes/rotations.
In a preferred embodiment the transformed adjustment factor from the machine vision system is provided to the robot controller. Based upon the robot control software application, the adjustment information is used to move the workpiece along each of the coordinates/rotations in which a difference is determined in order to position it closer to the actual desired position. The fiducial is again found, and compared, the process repeating until the difference is less than a predetermined minimum value near zero.
In one embodiment, the transformation function provides an under-correction with respect to actual difference to reduce the possibility of overshoot of the desired position during correction. The process therefore repeats, making continuously smaller position adjustments, until the desired workpiece position is essentially attained.
In addition, the robot can be commanded to accomplish two moves during each adjustment/movement cycle so as to approach a target from the same direction in order to accommodate gear backlash. The first motion command can, for example, instruct the robot to move the workpiece to the right of the desired/target position, then the second motion command instructs the workpiece to the desired/target position.
The workpiece can comprise a part to be moved from one location to a target location. Alternatively, the workpiece can be a tool that performs a job at the target location or any other object held by a robot arm that can be identified by the machine vision system based upon a distinct pattern or fiducial thereon.